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Abstract: In 2004 the Australian Flexible Learning Framework developed a suite of quantitative and qualitative indicators 
on the uptake, use and impact of e-learning in the Vocational Education and Training (VET) sector. These indicators 
were used to design items for a survey to gather quantitative data for benchmarking. A series of four surveys gathered 
data from VET providers, teachers, students and their employers. The data formed baseline indicators that were used to 
establish organisational goals and benchmarks for e-learning. These indicators were the first known set for 
benchmarking e-learning in Australia. 

The case studies in this paper illustrate ways in which VET providers have approached e-learning benchmarking, the 
benefits achieved and the lessons that they learned. The cases exemplify how VET providers have adapted the baseline 
indicators, how the indicators informed organisational plans and e-learning outcomes. The benefits of benchmarking are 
categorised under three purposes: reporting, performance management, and service improvement. A set of practical 
strategies is derived from the cases for consideration by other organisations interested in benchmarking e-learning 
services. 
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1. Introduction 

Over the last six years, the Australian 
Government has invested over $95m to enhance 
e-learning in the vocational education and training 
(VET) sector. This investment was based on the 
espoused benefits of e-learning. There is much 
anecdotal evidence and some research to 
corroborate the benefits of e-learning and provide 
support for the return on investment in e-learning 
technologies (see Block and Dobell, 1999; 
European Commission, 2000; Phillip, J., Phillip, P. 
and Zuniga, 2000; Roffe, 2002). However, there is 
variance in how such benefits are measured. 
Benchmarking for e-learning is very much in its 
infancy in Australia. In 2004, the Australian 
Flexible Learning Framework developed a set of 
twelve indicators to benchmark the uptake use 
and impact of e-learning in the VET sector. It was 
the intention of the Framework to develop the 
indicators, test these and make them available for 
users to adapt these to establish organisational 
goals and benchmarks for e-learning. The 
purpose was not to impose the benchmarking 
activity or use the indicators for comparison 
between institutions. Data against these indicators 
were used by VET institutions to assess the 
benefits of e-learning using the benchmarks at the 
organisational level, within their own contexts. A 
national dataset against the benchmark indicators 
was populated during 2005 and forms a baseline 
that illustrates trends in the uptake and impact of 
e-learning and the use of e-business services. 

This paper reports three case studies that 
illustrate ways in which VET providers have 
approached e-learning benchmarking, the benefits 
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they experienced and the lessons that they 
learned. The cases exemplify how VET providers 
have adapted the baseline indicators, how the 
indicators informed organisational plans and e- 
learning outcomes. Benefits of the benchmarking 
exercise served three purposes: reporting, 
performance measurement, and service 
improvement. A set of practical strategies is 
derived from the cases for consideration by other 
organisations interested in benchmarking e- 
learning services. 

2. e-Learning 

The last decade has seen a significant expansion 
in e-learning technologies for enhanced access to 
education and training in Australia, e-learning is 
conceptualised in a number of ways. Essentially, it 
is about the transmission of learning content using 
information technology and often refers to delivery 
using intra or Internet. The actual learning which 
involves identification of information, 
conceptualising and making meaning to enhance 
user’s knowledge base, understanding and skills, 
as well as finding the time and space for learning 
is left to the individual. Henry (2001) explains that 
the total e-learning solution comprises the 
integration of three elements: content, technology 
and services. His concept is also underpinned by 
the assumption that learners will be responsible 
for the cognitive tasks that will lead to learning. A 
leading researcher, Laurillard (2001), cautions 
that the way in which teaching is approached 
should be considered more important than the 
technology medium. This point is supported by 
Butson (2003) who stresses that technology is 
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limited in helping learners to understand how to 
think. 

There are compelling arguments for e-learning. 
“E-learning, when done well, can be as good or 
better than being in the classroom. It offers 
students a rich, compelling, and motivating 
experience” (Neal 2001). According to Roffe 
(2002) e-learning not only enhances access, but 
improves engagement, enhances learning, 
extends experiences in exploring, and empowers 
the learners to take responsibility for scheduling 
and managing the learning journey. His claims 
assume that the learner already has the skills and 
attributes to use the technology and adequately 
contextualise, integrate and apply the content to 
create new knowledge and understanding, and be 
transformed by the experience. Many 
organisations recognise the benefits of e-learning 
because it provides just-in-time, contemporary 
learning and can be accessed from any site using 
the right technology (Roffe, 2002). It is seen as a 
cost effective approach to facilitating learning to 
large groups using information and 
communication technology. The content could be 
personalised and is embedded in a learner 
centred framework. Many e-learning programs are 
interactive and can be updated rapidly. These and 
similar benefits were acknowledged in Young’s 
(2002) research on the first major benchmarking 
study of e-learning organisations in the United 
Kingdom. 

Initial investments in e-learning are costly, hence 
the performance, quality, usage, effectiveness 
and efficiency as a learning solution is of interest 
to many. However, the current research base 
informing evaluation of e-learning from a wide 
range of stakeholders or comprehensive return on 
investment remains limited. Despite the paucity in 
this field of research benchmarking exercises are 
used by organisations to define a level of 
performance, and identifying or establishing good 
practice to improve on that performance (Butson 
2003). According to Dublin (2004 p294) there are 
six fundamentals to ensure that e-learning is used 
by learners and embraced by the organisation. 
These fundamentals are premised on the 
understanding that e-learning is about: 

■ Business and providing a business solution; 

■ Providing a “return on expectation”, not just a 
return on investment; 

■ Enabling learning and driving performance, not 
training; 

■ People - learners, managers and executives - 
not technology; 

■ Motivating learners and energizing 
organisations; and 


■ Becoming invisible; interwoven into the very 
fabric of your organisation and its culture. 

The above are familiar to Ettinger, Holton and 
Blass’s (2005 p289) research with 29 companies 
who were e-learning pioneers. Ettinger et al. 
(2005) identified six key factors that underpinned 
e-learning: 

■ Delivering what the business needs 

■ Putting the learner at the heart of e-learning 

■ Providing high-quality content and technology 

■ Gaining support at senior levels for e-learning 

■ Providing pro-active support for e-learners 
(and their managers) through communication, 
promotion and marketing 

■ Creating an organisation that genuinely values 
learning. 

Most organisations implementing e-learning do so 
with a view to improving learning services, thereby 
achieving certain business goals (eg. Ettinger, 
Holton and Blass 2005, Dublin 2004, Roffe 2002, 
Young 2002). These organisations believe that 
improving learning services improves business 
outcomes. E-learning solutions have been known 
to support strategic outcomes (Fry, 2001). Many 
educational institutions seek e-learning solutions 
to maintain or enhance their market position in a 
highly competitive environment with declining 
public subsidy. E-learning services relates mainly 
to the management of e-learning as opposed to e- 
business. To distinguish between e-learning and 
e-business, the 2005 E-learning Benchmarking 
project adopted the following definitions: E- 
learning uses electronic media to deliver flexible 
vocational education and training. It includes 
access to, downloading and use of web, CD ROM 
or computer based learning resources in the 
classroom, workplace or home. It also includes 
online access to and participation in course 
activities (e.g. online simulations, online group 
discussions), directed use of the Internet for 
learning and research purposes, structured 
learning-based email communication and online 
assessment activities. E-learning does not include 
email dissemination of course information, email 
communication between a teacher/trainer and 
learner on a single learning issue, or online 
administration of learning activities. 

The following definition of e-business was 
adopted by the Benchmarking project: E-business 
services include client support and administrative 
services offered by training organisations that are 
delivered or supported by information and 
communications technologies. For example: 
online publication of general course information 
and relevant policies, regulations and strategies; 
online enrolment; online payments and electronic 
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forms; online access to student records; online 
library services; online information on student 
support services; online access to and delivery of 
student support services; and online access to 
results. 

The Australian Government considered e-learning 
a significant vehicle in transforming the VET 
business of teaching and training in more 
responsive ways. This became the key imperative 
for e-learning initiatives implemented through the 
Australian Flexible Learning Framework. 

3. E-learning in the Australian VET 
sector 

While information technology revolutionised the 
delivery of education and training, virtues of e- 
learning have principally modernised flexible 
delivery in the Australian VET sector. Investments 
in e-learning technologies were aimed at 
improving quality and access, fostering innovation 
and increasing flexibility in service provisions. 
Since the 1990s the Australian Flexible Learning 
Framework (Framework) has invested 
substantially and supported the uptake of e- 
learning through a range of national, state and 
organisational initiatives. Some initiatives provided 
professionals with access to the latest e-learning 
products and resources. To ensure that VET 
professionals are adequately equipped to meet a 
highly technology-driven learning environment the 
Framework has provided high quality professional 
development opportunities and support networks. 
Until 2005, no consistent sets of data were 
collected to assess the level of uptake and 
outcomes of e-learning in the VET sector. A 
benchmarking approach was undertaken to 
assess the return on investment in e-learning. 

4. Indicators for benchmarking e- 
learning 

Over 250 indicators for e-learning were identified 
in an environmental scan of Australian and 
international research and education agencies 
(Australian Flexible Learning Framework, 2004). 
However, literature surrounding benchmarking on 
e-learning is very limited. Interest in data about 
return on investment in e-learning in terms of 
uptake, use and outcomes on VET clients and 
providers inspired the Framework to develop and 
trial a set of 12 indicators that informed three 
areas of interest: 

Uptake and outcome of e-learning in the VET 
system 

■ % of VET unit enrolments that use e- 
learning. 

■ % of VET providers offering units that use 
e-learning. 


■ % of VET learners who through e-learning 
have increased skills and confidence in 
using information and communication 
technology (ICT). 

■ % of VET learners who through e-learning 
have or expect to have improved 
employment outcomes. 

■ % of VET clients who believe e-learning 
and e-business gave them flexibility in 
when, where and how they engaged with 
VET. 

■ % client satisfaction with e-learning 
experiences in VET. 

Uptake and impact of e-business 

■ % of VET providers offering e-business 
client, support and administrative services. 

■ % of VET clients using e-business client, 
support and administrative services offered 
by providers. 

■ % Client satisfaction with e-business 
experiences in VET. 

Uptake, use and outcomes of e-learning on VET 
teachers and trainers 

■ % of VET teachers/trainers delivering units 
that use e-learning. 

■ % of VET teachers/trainers who through e- 
learning have changed teaching practices 
in the design, development and delivery of 
units. 

■ % of VET teachers/trainers who believe 
increased access to e-learning resources 
has improved teaching and learning 
outcomes (Australian Flexible Learning 
Framework, 2005, p.6). 

5. Benchmarking survey 

National surveys using the 12 indicators were 
conducted with students, employers, training 
organisations, teachers and trainers across 
Australia. A convenience sampling approach was 
used for the national survey. Networks of the 
Framework were utilised to access the samples in 
each State (6) and Territory (2) in Australia. An 
online survey was completed by 1000 VET 
students from 100 training organisations across all 
States and Territories in Australia. The students 
represented public as well as private training 
organisations. Computer assisted telephone 
surveys were completed by 400 employers or 
their representatives from all States and 
Territories. Representatives from 400 Registered 
Training Organisations completed print based 
surveys. They represented 100 large, private and 
enterprise training providers and 200 small 
training organisations. One thousand VET 
teachers and trainers from 100 training 
organisations (public and private) across all 
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States and Territories supplied data by completing 
an online survey. The data and findings of the 
survey provided a baseline for VET stakeholders 
to benchmark their services and provisions and 
inform decision-making processes at 
organisational and State levels. (For further 
details see flexiblelearning.net.au/e- 

learningindicators). 

6. The case studies 

Three case studies reported in this paper 
exemplify how different institutions approached e- 
learning benchmarking, the benefits they achieved 
and lessons they learned. The cases focus on 
different purposes for benchmarking. The first 
case study relates to the process of conducting a 
survey with young learners aged 15-19 years and 
learners with a disability to collect data against 
identified indicators for benchmarking. The 
second study focused on collecting data for 
evaluation, monitoring and reporting to inform the 
Institute’s stakeholders (Board, managers, 
teachers, student support staff, administration 
staff and industry) about performance, planning, 
implementation and future directions for e- 
learning. The third case study concentrated on 
collecting data on the usage and quality of its 
online learning services to inform managers and 
teachers and to plan the next stage of 
development and improvements. The purpose of 
the case studies was to provide examples of how 
VET providers were organising benchmarking 
activities. A panel of the Framework, using a set 
of criteria, selected the sites. Rather than 
including well-established providers with 
significant advancements in e-learning, the panel 
sought sites that were relatively new to e-learning. 
This meant that providers with less established e- 
learning systems would be able to easily relate to 
the cases in this study. These examples were 
included to inspire other providers to engage in e- 
learning benchmarking. Participation in the case 
studies was voluntary. 

The case studies in this paper focused more on 
the management of e-learning as opposed to e- 
business. Each case study ‘aligned’ it’s 
benchmarking exercise against the indicators 
developed by the Framework. As such, the 
indicators and data sets are not exactly the same, 
however, have a degree of comparability. 
Considering the distinct peculiarity of each site, it 
would be erroneous to make any direct 
comparisons using the quantitative data against 
the indicators. Quantitative data collected by each 
site remained commercial-in-confidence. Data for 
the three case studies reported in this paper was 
collected from face-to-face and telephone 
interviews with the project managers and their e¬ 


learning support staff in each site. Data for the 
first case study was gathered from the project 
reports and interviews with two project managers. 
The second case study was prepared using data 
from the project documents (plans and reports) 
and interviews with three staff at the training 
organisation. Data for the third case study was 
gathered from the project report and an interview 
with the project manager. 

6.1 Case study 1: e-Learning for target 
learner groups - youth and learners 
with a disability 

The 2005 e-learning for Target Learner Groups 
(ELTLG) focused on young people in the 15-19 
age group (including VET in Schools, disengaged 
youth and school-based apprenticeships), and 
learners with a disability. The main aim was to 
improve employment-related training 
opportunities and employment skills through the 
use of appropriate e-learning programs and 
resources. This case study trialed the survey 
process with young learners in selected 
organisations spread across Western Australia, 
New South Wales, Victoria, South Australia and 
Queensland. Fifty-two youths and fifty-four 
students with a disability responded to items on 
the survey which was made available on-line. The 
URL address for the survey was provided to 
trainers in each of the participating states to 
encourage their students to complete the survey. 

6.1.1 Benefits of the survey 

The datasets for this survey verified anecdotal 
data about young learners and those with a 
disability. Quantitative data was presented to 
managers to make informed decisions about e- 
learning services for young learners and learners 
with a disability. The data informed decision 
making for organisational planning and allocation 
of resources. Knowledge about how e-learning 
tools and technologies can be better used to 
assist learners is essential to improve services 
and course delivery to enable young learners and 
those with a disability to participate fully and 
equally in VET. When combined with data from 
other sources such as the networks and intranet 
usage statistics, a holistic picture of the service 
provisions emerged. The data enabled teachers to 
monitor, evaluate and reflect on the impact of 
changes in teaching strategies and choice of 
content and resources. Teachers gained 
confidence from the positive responses from 
students about e-learning tools and technologies. 
Benchmarking data provided support staff with 
information about communication and support 
strategies that involve e-learning tools and 
technologies, and the use and appropriateness of 
these services. The information is then used for 
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future planning and development of e-support 
services. 

6.1.2 Lessons about the survey process 

The trialling of the survey presented several 
learning experiences that can be used to improve 
the process and achieve a better return rate. 

“We expected this to be an easy task, but 
learned that just placing the survey online 
and expecting students to fill them out does 
not get us far,” explained Bruce Enting, 
project manager. 

The project highlighted the importance of detailed 
planning, networking and regularly keeping in 
touch with those assisting with administering the 
survey. There was evidence of low literacy 
problems levels among some young learners and 
these presented problems with reading and 
interpreting the survey items. The survey with 
youths noted some variation in the way e-learning 
and e-business were conceptualised. Not all 
students participating in e-learning had regular 
access to technology or to the internet. If it is 
available only during the contact hours or only in 
classrooms, a time period needs to be negotiated 
with the survey administrators to allow participants 
to complete the survey during these times. The 
context and environments in which e-learning for 
young learners takes place is diverse. 

6.2 Case study 2: Building sustainability 
- performance indicators for 
educational delivery 

E-business systems and facilities at the Hunter 
Institute of TAFE were put in place to facilitate and 
serve organisational business goals. e-Learning 
structures support blended learning which is most 
appropriate for learners from the catchment 
served by the Institute. Together, the two were set 
up to provide quality client services that contribute 
to sustainable regional development. 
Benchmarking is used to measure how e- 
business and e-learning impact on the institute’s 
service delivery. The indicators are used to 
collect data for evaluation, monitoring and 
reporting to inform the Institute, its Board and 
other stakeholders about performance, 
implementation issues and future directions. The 
set of key indicators used by Hunter Institute 
maintained a balance between practicality in 
collection and analysis, usability of the data, and 
costs. The process is inclusive of all stakeholders 
including the Board, managers, teachers, student 
support staff and the local community. The 
institute is mindful of its learners and their 
communities’ readiness to embrace technology 
and e-learning. Hence decisions informed by the 
benchmarking data consider the context of the 


learners, and the local communities and 
industries. Data sets against the agreed indicators 
are analysed at multiple levels to report on 
findings at the Institute, faculty, unit and team 
levels. The data sets offer options for: 

■ Monitoring performance using time series data 

■ Comparisons with other TAFE institutes in 
New South Wales 

■ Comparisons with other registered training 
organisations and best practice examples 
nationally and internationally. 

The institute does not compare the data sets 
between faculties largely because of the diversity 
in the student cohorts and their learning needs. 
Data sets against the performance indicators 
would be supplemented with data from evaluative 
case studies to obtain more contextual 
understanding about performance measures and 
the related outcomes. 

6.2.1 Benefits experienced by the institute: 

“The potential benefits of the data are 
recognised by some stakeholders who 
expressed interest in more data to make 
informed decisions. Others need more time 
to come on board,” said Louise King, 
Director, Educational Development. 

The data verified anecdotal information; tested 
assumptions held by the stakeholders and 
informed economic decisions about resources and 
efficiencies. The benchmarking data informed 
decision-making at various levels by different 
internal stakeholders such as teachers, 
administrators, student support and management 
staff. At an organisational level, the data sets 
helped monitor performance against the 
Australian Business Excellence Framework 
standards and assisted with setting targets to 
exceed the average ratings of the standards, as 
well as monitor progress against agreed 
measures and goals. Data for the Hunter Institute 
is added to the central database for the whole of 
the TAFE New South Wales (NSW) network. The 
Educational Development unit provides data on a 
quarterly basis, enabling each team to monitor 
progress against each indicator. The survey data 
informs performance at the team level and 
provides recognition and encouragement to 
improve services and innovation. Teams that 
show improvements are recognised by their 
managers and the Institute. News of their 
outstanding achievements is shared with other 
staff and key industry partners who support their 
activities. The local community and industries that 
supports the activities of the institute recognise 
the Institute’s progress and support further 
improvements. “This is important for our 
relationship with local industries and communities. 
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They all show pride in the achievements of the 
Institute,” explained Louise King, Director, 
Educational Development. The project recognised 
the potential in undertaking benchmarking to 
improve products, content, and service 
development, professional development, change 
management for e-learning, IT planning, business 
planning and marketing. 

6.2.2 Lessons from the impact of e-business 
and e-learning on service delivery: 

Benchmarking takes a lot of time and planning. 
“One must not assume that the student support 
officer will have time to collect data on top of 
his/her normal duties,” advised Louise King, 
Director, Educational Development. It is critical to 
be inclusive of all stakeholders to ensure progress 
at all levels without expecting everyone to come 
on board from the start. “Some individuals and 
teams need time and space. It is best to focus and 
start with those who are keen and ready,” 
observed Louise King. Through this project, the 
survey team learned that a phased approach to 
cultural change at the organisational and 
community levels would improve engagement and 
commitment to the benchmarking exercise and 
goals of improving e-learning services. 

7. Case study 3: Evaluation of 
Learnline 

Charles Darwin University (CDU) is a dual sector 
institution with approximately 17,000 students and 
nine campuses located across the Northern 
Territory. Web-based learning was formally 
introduced in 2003 using a learning management 
system, called Learnline. Learnline integrates 
web-based learning, other e-learning resources 
and student administration data into a centrally 
managed system that is accessed by staff and 
students. The system is based on licensed 
Blackboard software and includes Horizon Wimba 
LiveClassroom and Voice Tools plug-ins, locally 
developed websites and blogs. Since the 
implementation of Learnline, the University has 
experienced a large increase in enrolments in 
units offered on-line. A very diverse group of 
learners spread across urban, regional and 
remote areas were able to access learning and 
training using Learnline. On-line delivery of higher 
education units were steadily increasing, but 
Vocational Education and Training units needed 
more input and improvements to encourage the 
move to blended and on-line delivery modes. 

Prior to the evaluation of Learnline, there was no 
hard data available to show what helped or 
hindered staff moving to online delivery. Evidence 
about the usage and quality of its online learning 


services was needed. The university provided 
strategic support to carry out the exercise and 
funded an external consultant to undertake a 
survey of staff who were both users and non¬ 
users of Learnline. This initiative and support led 
to increased staff interest, support and 
participation in the evaluation. Eighty teachers 
(approx. 15% of academic staff) participated in the 
survey. The survey used a combination of focus 
groups, telephone and face-to-face interviews, 
and email surveys for data gathering. The 
response rate was high (more than 80%). 

7.1.1 Benefits of the evaluation of Learnline 

Data highlighted the specific areas of e-learning 
services that were considered satisfactory and 
those that needed further improvement. For 
example, the existing introductory staff training 
sessions for Learnline, were rated highly. On the 
other hand customised training in some subject 
areas, for some of the less commonly used 
features of Learnline, and for sessional and 
remote area staff were identified. The information 
helped with the review of professional 
development content and delivery and in shaping 
strategies for equipping teachers with the right 
sets of knowledge and skills so that the quality of 
online content and services to students could be 
improved. The data, which provided solid 
evidence on the performance of Learnline, was 
submitted to management to provide justification 
for additional resources such as IT hardware and 
software, and for staff time. This aligned well with 
the evidence-based framework that the university 
uses when making decisions. The findings gave 
an understanding of the advantages and 
disadvantages of collecting data against the 
indicators. There was much interest in the 
outcomes from teachers, CDU management and 
external networks of the university who were 
considering similar exercises. The university is 
now able to extract data on the use of e-learning 
in terms of the number units and access to those 
units. It now has data on a set of 24 indicators (3 
on teaching and learning and 21 on course 
development, institutional support, course 
structure, faculty support, student support, and 
evaluation and assessment) that benchmark the 
quality of web-based learning resources. The 
findings of the survey inform and align with 
Northern Territory’s Flexible Learning Plan and 
the Australian Flexible Learning Framework. 

7.1.2 Lessons from the evaluation of 
Learnline 

The findings highlighted the need to allocate 
teaching staff with more time for the development 
of the online content, especially for new courses 
and units, or those undergoing major revisions. 
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8. Key benefits of e-learning 
benchmarking 

All three case studies recognised the potential in 
undertaking benchmarking to improve products, 
content and service development, professional 
development, change management for e-learning, 
IT planning, business planning and marketing. 
Key benefits of undertaking e-learning 
benchmarking, as experienced by the case 
studies, are listed under three broad purposes: 
reporting framework, performance measurement 
and service improvement. 

8.1 Reporting framework 

E-learning benchmarking data provides real 
evidence that informs an evidence based decision 
making framework. Benchmarking datasets 
contribute to reporting frameworks by verifying 
anecdotal data and testing of assumptions held by 
the stakeholders. They provide real evidence of 
performance against the indicators. The datasets 
inform decision making at various levels by 
different internal stakeholders such as teachers, 
administrators, student support and management 
staff. They provide substantiate justification for 
additional resources such as IT hardware and 
software, and for staff time. Datasets inform 
decision making for organisational planning, 
efficiencies, effectiveness and allocation of 
resources. When combined with data from other 
sources a more holistic picture of the service 
provisions develops. Benchmarking data could be 
added to a larger database for future data mining 
by central agencies as well as other contributing 
organisations for comparisons and drawing 
evidence based conclusions. The datasets inform 
and align with strategic directions of the 
organisation, as well as State and National goals 
for e-learning. 

8.2 Performance measurement 

Monitoring progress using performance 
measurement is a common approach in all 
organisations. Datasets from benchmarking could 
be used to help monitor performance against 
existing standards, and assist with setting targets 
to exceed existing ratings of the standards, as 
well as monitor progress against agreed 
measures and goals. They inform performance at 
the team level and could contribute to internal 
recognition and reward systems. Outstanding 
performance stories could be shared with key 
stakeholders, advocates and supporting networks. 
Dissemination of high success and achievements 
increases staff confidence. Datasets provide a 
bearing on how particular groups of learners 
compare with others regarding common sets of 
services. 


8.3 Service improvement 

Analysis of benchmarking datasets could identify 
service areas that are highly regarded by clients 
and those that are limited and in need for 
improvement. Benchmarking data identifies 
specific areas for improvement to enhance 
services and course delivery. They enable 
teachers to monitor, evaluate and reflect on the 
impact of changes in teaching strategies and 
choice of content and resources. Datasets provide 
support staff with information about 
communication and support strategies that involve 
e-learning tools and technologies, and the use 
and appropriateness of these services. The 
information is then used for future planning and 
development of e-support services. Benchmarking 
data helps review professional development 
content and delivery and shape strategies for 
equipping teachers with the right sets of 
knowledge and skills so that the quality of online 
content and services to students could be 
improved. Datasets provide a point of reference to 
improve and be competitive as well as innovative 
in educational delivery. 

9. Suggested strategies for e-learning 
benchmarking 

The following strategies are drawn from lessons 
that the three case studies highlighted. They are 
grouped under three main actions for 
benchmarking: data collection, data interpretation 
and implementation of e-learning. 

10. Data collection 

■ Allow flexibility in the way the survey is 
conducted to collect data. For those who do 
not have access to technology, hard copies of 
the survey should be supplied to be completed 
and posted back to the survey administrators. 
Some students being surveyed may like to 
complete these in their own time and place, 
others may like to do it in class with the 
teacher/trainer assisting with reading and 
interpreting the items. 

■ Support for those with literacy problems must 
be made available. 

■ Key terms such as e-learning and e-business 
should be defined clearly. Include examples of 
what constitutes e-learning and e-business. 

■ A supportive relationship with the trainers or 
teachers (administrators) of the survey would 
ensure improved response rates. 

11. Data interpretation 

Data must be interpreted within the contexts of the 
team, faculty and the organisation, to draw valid 
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conclusions or make comparisons. Interpretation 
of data may get clouded by focusing on 
embracing technology for e-learning. Serious 
consideration must also be given to client and 
staff capability and capacities to take up and use 
e-learning. 

12. Implementation of e-learning 
benchmarking 

■ A clear change management process that 
considers the changing roles and 
responsibilities of all stakeholders will minimise 
confusion about task allocations and timelines 
for collecting, analysing, interpreting and 
disseminating data. A phased approach to 
introducing and implementing e-learning 
benchmarking would ease the cultural change. 
When planning for benchmarking managers 
need to engage and work in partnership with 
all stakeholders including IT, HR and 
management staff as active players, not just 
adversaries. 

■ Clearly defined goals and timelines should be 
communicated to all stakeholders so that they 
all know what and when to expect. Both, a 
change communications plan and a marketing 
communication plan are needed (Dublin 2004). 

■ Invest adequate time in planning and 
development of e-learning content. 

■ Include a framework of recognition and reward. 
According to Ettinger et al. (2005) a positive 
framework of recognition and reward enhances 
motivation. 

The above strategies form a guide for others 
interested in undertaking benchmarking exercise 
to improve e-learning services. 


13. Summary 

The case studies in this paper illustrate ways in 
which VET providers have approached e-learning 
benchmarking, the benefits achieved and the 
lessons that they learned. These cases exemplify 
how VET providers in Australia have adapted the 
baseline indicators, how the indicators informed 
organisational plans and e-learning outcomes. 
The data sets obtained by the three case study 
sites informed three main organisational 
purposes: reporting, performance measurement, 
and service improvements. The benchmarking 
datasets provided evidence on current 
performance and progress as well as 
improvements in products, content and service 
development, professional development, change 
management for e-learning, IT planning, business 
planning and marketing. The experiences from the 
benchmarking exercises highlighted a set of 
strategies for others interested in e-learning 
benchmarking to gain the most benefits. Other 
users could adopt strategies for data collection, 
data interpretation and implementation of e- 
learning benchmarking at the organisational. 
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